Methods and Systems for Detecting Sensor Occlusions

ABSTRACT

One example method involves rotating a housing of a light detection and ranging (LIDAR) device about a first axis. The housing includes a first optical window and a second optical window. The method also involves transmitting a first plurality of light pulses through the first optical window to obtain a first scan of a field-of-view (FOV) of the LIDAR device. The method also involves transmitting a second plurality of light pulses through the second optical window to obtain a second scan of the FOV. The method also involves identifying, based on the first scan and the second scan, a portion of the FOV that is at least partially occluded by an occlusion.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/755,011 filed on Nov. 2, 2018, the entirety of which isincorporated herein by reference.

BACKGROUND

Active sensors, such as light detection and ranging (LIDAR) sensors,radio detection and ranging (RADAR) sensors, sound navigation andranging (SONAR) sensors, among others, are sensors that can scan asurrounding environment by emitting signals toward the surroundingenvironment and detecting reflections of the emitted signals.

For example, a LIDAR sensor can determine distances to environmentalfeatures while scanning through a scene to assemble a “point cloud”indicative of reflective surfaces in the environment. Individual pointsin the point cloud can be determined, for example, by transmitting alaser pulse and detecting a returning pulse, if any, reflected from anobject in the environment, and then determining a distance to the objectaccording to a time delay between the transmission of the pulse and thereception of the reflected pulse. As a result, for example, athree-dimensional map of points indicative of locations of reflectivefeatures in the environment can be generated.

SUMMARY

In one example, a method is provided. The method involves rotating ahousing of a light detection and ranging (LIDAR) device. The housingincludes a first optical window and a second optical window. The methodalso involves transmitting a first plurality of light pulses through thefirst optical window to obtain a first scan of a field-of-view (FOV) ofthe LIDAR device. The method also involves transmitting a secondplurality of light pulses through the second optical window to obtain asecond scan of the FOV. The method also involves identifying, based onthe first scan and the second scan, a portion of the FOV that is atleast partially occluded by an occlusion.

In another example, a light detection and ranging (LIDAR) device isprovided. The LIDAR device includes a housing that has a first opticalwindow and a second optical window. The LIDAR device also includes anactuator configured to rotate the housing. The LIDAR device alsoincludes a controller that causes the LIDAR device to performoperations. The operations comprise transmitting a first plurality oflight pulses through the first optical window and a second plurality oflight pulses through the second optical window. The operations alsocomprise obtaining a first scan of a field-of-view (FOV) of the LIDARdevice based on the transmitted first plurality of light pulses, and asecond scan of the FOV based on the transmitted second plurality oflight pulses. The operations also comprise identifying, based on thefirst scan and the second scan, a portion of the FOV that is at leastpartially occluded by an occlusion.

In yet another example, a method is provided. The method involvesreceiving, from a light detection and ranging (LIDAR) device, dataindicative of a first scan of a field-of-view (FOV). The first scan isbased on a first plurality of light pulses transmitted through a firstoptical window of the LIDAR device. The method also involves receivingdata indicative of a second scan of the FOV from the LIDAR device. Thesecond scan is based on a second plurality of light pulses transmittedthrough a second optical window of the LIDAR device. The method alsoinvolves identifying, based on the first scan and the second scan, aportion of the FOV that is at least partially occluded by an occlusion.

In still another example, a system is provided. The system comprisesmeans for rotating a housing of a light detection and ranging (LIDAR)device. The housing includes a first optical window and a second opticalwindow. The system also comprises means for transmitting a firstplurality of light pulses through the first optical window to obtain afirst scan of a field-of-view (FOV) of the LIDAR device. The system alsocomprises means for transmitting a second plurality of light pulsesthrough the second optical window to obtain a second scan of the FOV.The system also comprises means for identifying, based on the first scanand the second scan, a portion of the FOV that is at least partiallyoccluded by an occlusion.

In still another example, a system is provided. The system comprisesmeans for receiving, from a light detection and ranging (LIDAR) device,data indicative of a first scan of a field-of-view (FOV). The first scanis based on a first plurality of light pulses transmitted through afirst optical window of the LIDAR device. The system also comprisesmeans for receiving data indicative of a second scan of the FOV from theLIDAR device. The second scan is based on a second plurality of lightpulses transmitted through a second optical window of the LIDAR device.The system also comprises means for identifying, based on the first scanand the second scan, a portion of the FOV that is at least partiallyoccluded by an occlusion.

These as well as other aspects, advantages, and alternatives will becomeapparent to those of ordinary skill in the art by reading the followingdetailed description with reference where appropriate to theaccompanying drawings. Further, it should be understood that thedescription provided in this summary section and elsewhere in thisdocument is intended to illustrate the claimed subject matter by way ofexample and not by way of limitation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simplified block diagram of a system, according to exampleembodiments.

FIG. 2A illustrates a LIDAR device, according to example embodiments.

FIG. 2B illustrates a partial perspective view of the LIDAR device.

FIG. 2C illustrates a partial cross-section view of the LIDAR device.

FIG. 2D illustrates another partial cross-section view of the LIDARdevice.

FIG. 3 is a simplified block diagram of a vehicle, according to exampleembodiments.

FIG. 4 is a flowchart of a method, according to example embodiments.

FIG. 5 is a flowchart of another method, according to exampleembodiments.

FIG. 6 is a flowchart of yet another method, according to exampleembodiments.

FIG. 7 is a flowchart of still another method, according to exampleembodiments.

DETAILED DESCRIPTION

Exemplary implementations are described herein. It should be understoodthat the word “exemplary” is used herein to mean “serving as an example,instance, or illustration.” Any implementation or feature describedherein as “exemplary” or “illustrative” is not necessarily to beconstrued as preferred or advantageous over other implementations orfeatures. In the figures, similar symbols typically identify similarcomponents, unless context dictates otherwise. The exampleimplementations described herein are not meant to be limiting. It willbe readily understood that the aspects of the present disclosure, asgenerally described herein and illustrated in the figures, can bearranged, substituted, combined, separated, and designed in a widevariety of different configurations.

I. OVERVIEW

In some scenarios, a FOV of an active sensor may become occluded (atleast partially) by objects, obstacles, debris, dirt, scratches,deformations, and/or other types of occlusions. An occlusion may preventone or more signals (or portions thereof) transmitted by the activesensor from reaching regions of the environment that are behind theocclusion, and/or prevent one or more signals (or portions thereof)propagating from the environment from being received by the activesensor. Some types of occlusions may be physically separated from (e.g.,not attached to) the active sensor. Other types of occlusions may bedisposed on or otherwise attached to the active sensor (e.g.,accumulated dirt or other debris on the active sensor, etc.).

Some example implementations herein may relate to detection of sensorocclusions and/or system operations associated with the detection ofsuch occlusions.

One example LIDAR device herein includes a housing that has a firstoptical window and a second optical window. The LIDAR device may beconfigured to transmit a first plurality of light pulses through thefirst optical window and a second plurality of light pulses through thesecond optical window. The second optical window may be located at anopposite side of the housing from the first optical window. In this way,the LIDAR device can obtain a first scan of the FOV based on the firstplurality of light pulses (transmitted through the first optical window)and a second scan of the same FOV based on the second plurality of lightpulses (transmitted through the second optical window). The LIDAR devicemay also be configured to rotate the housing. In one implementation, theLIDAR device may scan a horizontal FOV of 360 degrees by rotating thehousing a complete rotation (e.g., 360 degrees) about a yaw axis of theLIDAR device. In alternative implementations, the LIDAR device may beconfigured to scan a different horizontal FOV (e.g., by rotating thehousing less than 360 degrees about the yaw axis, etc.).

In some implementations, the LIDAR device could use the first and secondoptical windows to obtain two scans of the same FOV simultaneouslyduring a single complete (360 degree) rotation of the housing. With thisarrangement, the first scan and the second scan may be generally similarto one another due to an overlap of the respective scanning time periodsand the respective FOVs associated with each scan. Accordingly, in someexamples, the first scan can be used as a reference for validating thesecond scan, and vice versa. For example, by comparing the two scans,the LIDAR device could distinguish between occlusions that affect onlyone optical window (e.g., dirt or debris attached to the optical window,etc.) from occlusions that affect both optical windows (e.g., nearbyobject, tree, wall, etc.).

In some examples, the LIDAR device can detect the occlusion based on oneor more scans obtained through one of the two optical windows. In oneexample, the LIDAR device may keep track of the numbers of light pulsestransmitted toward and/or received from respective sections of the firstoptical window. In this example, the LIDAR device may then determinewhether an occlusion that occludes a portion of the FOV associated witha particular section of the first optical window is potentially present,and/or whether the occlusion is coupled to the LIDAR device (e.g., dustdisposed on the particular section, etc.). For instance, if thedifference between a number of emitted light pulses transmitted towardthe particular section and a number of received light pulses through theparticular section increases (or exceeds a threshold), then the LIDARdevice (or a computing system coupled to the LIDAR device) coulddetermine that an occlusion may be potentially present on or near thatparticular section of the optical window. Alternatively or additionally,in another example, the LIDAR device (or the computing system) may beconfigured to detect the occlusion by monitoring light pulsestransmitted and/or received through a plurality of sections of thesecond optical window.

In some examples, the LIDAR device may alternatively include a singleoptical window (instead of including both the first optical window andthe second optical window). In these examples, the LIDAR device mayobtain one or more scans of the environment through the single opticalwindow (e.g., during rotation of the LIDAR device about an axis, etc.).To that end, some examples herein may involve detecting the occlusion(and/or determining whether the occlusion is physically coupled to theLIDAR device) based on one or more scans obtained through the singleoptical window of the LIDAR device, in line with the discussion above.

Depending on the type of occlusion, in some examples, the LIDAR device(or a system that uses the LIDAR device) could decide how to respond tothe detection of the occlusion. In a first example, where the occlusionis determined to occlude only one optical window (e.g., dirt, dust,etc.), the LIDAR device may activate a cleaning mechanism (e.g., liquidspray, high-pressure gas tube, wiper, etc.) to attempt removal of theocclusion. In a second example, where the occlusion is determined toocclude both optical windows (e.g., a plastic bag covering a portion ofthe LIDAR device, etc.), the LIDAR device may decide to wait for a givenamount of time (or a given number of rotations of the housing) for theocclusion to be removed (e.g., an occluding plastic bag may be blownaway by wind), or alert a system that uses the LIDAR that the portion ofthe FOV is occluded, among other possibilities.

II. EXAMPLE SYSTEMS AND DEVICES

A non-exhaustive list of example sensors of the present disclosureincludes LIDAR sensors, RADAR sensors, SONAR sensors, active IR cameras,and/or microwave cameras, among others. To that end, some examplesensors herein may include active sensors that emit a signal (e.g.,visible light signal, invisible light signal, radio-frequency signal,microwave signal, sound signal, etc.), and then detect reflections ofthe emitted signal from the surrounding environment.

FIG. 1 is a simplified block diagram of a system 100, according toexample embodiments. As shown, system 100 includes a power supplyarrangement 102, a controller 104, a rotating platform 110, a stationaryplatform 112, one or more actuators 114, one or more encoders 116, arotary link 118, a transmitter 120, a receiver 130, one or more opticalelements 140, a housing 150, and one or more cleaning apparatuses 160.In some embodiments, system 100 may include more, fewer, or differentcomponents. Additionally, the components shown may be combined ordivided in any number of ways.

Power supply arrangement 102 may be configured to supply, receive,and/or distribute power to various components of system 100. To thatend, power supply arrangement 102 may include or otherwise take the formof a power source (e.g., battery cells, etc.) disposed within system 100and connected to various components of the system 100 in any feasiblemanner, so as to supply power to those components. Additionally oralternatively, power supply arrangement 102 may include or otherwisetake the form of a power adapter configured to receive power from one ormore external power sources (e.g., from a power source arranged in avehicle to which system 100 is mounted) and to transmit the receivedpower to various components of system 100.

Controller 104 may include one or more electronic components and/orsystems arranged to facilitate certain operations of system 100.Controller 104 may be disposed within system 100 in any feasible manner.In one embodiment, controller 104 may be disposed, at least partially,within a central cavity region of rotary link 118.

In some examples, controller 104 may include or otherwise be coupled towiring used for transfer of control signals to various components ofsystem 100 and/or for transfer of data from various components of system100 to controller 104. For example, the data that controller 104receives may include sensor data indicating detections of signals byreceiver 130, among other possibilities. Moreover, the control signalssent by controller 104 may operate various components of system 100,such as by controlling emission of signals by transmitter 120,controlling detection of signals by the receiver 130, and/or controllingactuator(s) 114 to rotate rotating platform 110, among otherpossibilities.

As shown, controller 104 may include one or more processors 106 and datastorage 108. In some examples, data storage 108 may store programinstructions executable by processor(s) 106 to cause system 100 toperform the various operations described herein. To that end,processor(s) 106 may comprise one or more general-purpose processorsand/or one or more special-purpose processors. To the extent thatcontroller 104 includes more than one processor, such processors couldwork separately or in combination. In some examples, data storage 108may comprise one or more volatile and/or one or more non-volatilestorage components, such as optical, magnetic, and/or organic storage,and data storage 108 may be optionally integrated in whole or in partwith the processor(s).

In some examples, controller 104 may communicate with an externalcontroller or the like (e.g., a computing system arranged in a vehicleto which system 100 is mounted) so as to help facilitate transfer ofcontrol signals and/or data between the external controller and thevarious components of system 100. Additionally or alternatively, in someexamples, controller 104 may include circuitry wired to perform one ormore of the operations described herein. For example, controller 104 mayinclude one or more pulser circuits that provide pulse timing signalsfor triggering emission of pulses or other signals by transmitter 120.Additionally or alternatively, in some examples, controller 104 mayinclude one or more special purpose processors, servos, or other typesof controllers. For example, controller 104 may include aproportional-integral-derivative (PID) controller or other control loopfeedback mechanism that operates actuator(s) 114 to cause the rotatingplatform to rotate at a particular frequency or phase. Other examplesare possible as well.

Rotating platform 110 may be configured to rotate about an axis. To thatend, rotating platform 110 can be formed from any solid materialsuitable for supporting one or more components mounted thereon. Forexample, transmitter 120 and receiver 130 may be arranged on rotatingplatform 110 such that each of these components moves relative to theenvironment based on rotation of rotating platform 110. In particular,these components could be rotated about an axis so that system 100 mayobtain information from various directions. For instance, where the axisof rotation is a vertical axis, a pointing direction of system 100 canbe adjusted horizontally by actuating the rotating platform 110 aboutthe vertical axis.

Stationary platform 112 may take on any shape or form and may beconfigured for coupling to various structures, such as to a top of avehicle, a robotic platform, assembly line machine, or any other systemthat employs system 100 to scan its surrounding environment, forexample. Also, the coupling of the stationary platform may be carriedout via any feasible connector arrangement (e.g., bolts, screws, etc.).

Actuator(s) 114 may include motors, pneumatic actuators, hydraulicpistons, and/or piezoelectric actuators, and/or any other types ofactuators. In one example, actuator(s) 114 may include a first actuatorconfigured to actuate the rotating platform 110 about the axis ofrotation of rotating platform 110. In another example, actuator(s) 114may include a second actuator configured to rotate one or morecomponents of system 100 about a different axis of rotation. Forinstance, the second actuator may rotate an optical element (e.g.,mirror, etc.) about a second axis (e.g., horizontal axis, etc.) toadjust a direction of an emitted light pulse (e.g., vertically, etc.).In yet another example, actuator(s) 114 may include a third actuatorconfigured to tilt (or otherwise move) one or more components of system100. For instance, the third actuator can be used to move or replace afilter or other type of optical element 140 along an optical path of anemitted light pulse, or can be used to tilt rotating platform (e.g., toadjust the extents of a field-of-view (FOV) scanned by system 100,etc.), among other possibilities.

Encoder(s) 116 may include any type of encoder (e.g., mechanicalencoders, optical encoders, magnetic encoders, capacitive encoders,etc.). In general, encoder(s) 116 may be configured to providerotational position measurements of a device that rotates about an axis.In one example, encoder(s) 116 may include a first encoder coupled torotating platform 110 to measure rotational positions of platform 110about an axis of rotation of platform 110. In another example,encoder(s) 116 may include a second encoder coupled to a mirror (orother optical element 140) to measure rotational positions of the mirrorabout an axis of rotation of the mirror.

Rotary link 118 directly or indirectly couples stationary platform 112to rotating platform 110. To that end, rotary link 118 may take on anyshape, form and material that provides for rotation of rotating platform110 about an axis relative to the stationary platform 112. For instance,rotary link 118 may take the form of a shaft or the like that rotatesbased on actuation from actuator(s) 114, thereby transferring mechanicalforces from actuator(s) 114 to rotating platform 110. In oneimplementation, rotary link 118 may have a central cavity in which oneor more components of system 100 may be disposed. In some examples,rotary link 118 may also provide a communication link for transferringdata and/or instructions between stationary platform 112 and rotatingplatform 110 (and/or components thereon such as transmitter 120 andreceiver 130).

Transmitter 120 may be configured to transmit signals toward anenvironment of system 100. As shown, transmitter 120 may include one ormore emitters 122. Emitters 122 may include various types of emittersdepending on a configuration of system 100.

In a first example, where system 100 is configured as a LIDAR device,transmitter 120 may include one or more light emitters 122 that emit oneor more light beams and/or pulses having wavelengths within a wavelengthrange. The wavelength range could be, for example, in the ultraviolet,visible, and/or infrared portions of the electromagnetic spectrum. Insome examples, the wavelength range can be a narrow wavelength range,such as that provided by lasers. A non-exhaustive list of example lightemitters 122 includes laser diodes, diode bars, light emitting diodes(LED), vertical cavity surface emitting lasers (VCSEL), organic lightemitting diodes (OLED), polymer light emitting diodes (PLED), lightemitting polymers (LEP), liquid crystal displays (LCD),microelectromechanical systems (MEMS), fiber lasers, and/or any otherdevice configured to selectively transmit, reflect, and/or emit light toprovide a plurality of emitted light beams and/or pulses.

In a second example, where system 100 is configured as an activeinfrared (IR) camera, transmitter 120 may include one or more emitters122 configured to emit IR radiation to illuminate a scene. To that end,transmitter 120 may include any type of emitter (e.g., light source,etc.) configured to provide the IR radiation.

In a third example, where system 100 is configured as a RADAR device,transmitter 120 may include one or more antennas, waveguides, and/orother type of RADAR signal emitters 122, that are configured to emitand/or direct modulated radio-frequency (RF) signals toward anenvironment of system 100.

In a fourth example, where system 100 is configured as a SONAR device,transmitter 120 may include one or more acoustic transducers, such aspiezoelectric transducers, magnetostrictive transducers, electrostatictransducers, and/or other types of SONAR signal emitters 122, that areconfigured to emit a modulated sound signal toward an environment ofsystem 100. In some implementations, the acoustic transducers can beconfigured to emit sound signals within a particular wavelength range(e.g., infrasonic, ultrasonic, etc.). Other examples are possible aswell.

In some implementations, system 100 (and/or transmitter 120) can beconfigured to emit a plurality of signals (e.g., light beams, IRsignals, RF waves, sound waves, etc.) in a relative spatial arrangementthat defines a FOV of system 100. For example, each beam (or signal) maybe configured to propagate toward a portion of the FOV. In this example,multiple adjacent (and/or partially overlapping) beams may be directedto scan multiple respective portions of the FOV during a scan operationperformed by system 100. Other examples are possible as well.

Receiver 130 may include one or more detectors 132 configured to detectreflections of the signals emitted by transmitter 120.

In a first example, where system 100 is configured as a RADAR device,receiver 130 may include one or more antennas (i.e., detectors 132)configured to detect reflections of the RF signal transmitted bytransmitter 120. To that end, in some implementations, the one or moreantennas of transmitter 120 and receiver 130 can be physicallyimplemented as the same physical antenna structures.

In a second example, where system 100 is configured as a SONAR device,receiver 130 may include one or more sound sensors 110 (e.g.,microphones, etc.) that are configured to detect reflections of thesound signals emitted by transmitter 120.

In a third example, where system 100 is configured as an active IRcamera, receiver 130 may include one or more light detectors 132 (e.g.,charge-coupled devices (CCDs), etc.) that are configured to detect asource wavelength of IR light transmitted by transmitter 120 andreflected off a scene toward receiver 130.

In a fourth example, where system 100 is configured as a LIDAR device,receiver 130 may include one or more light detectors 132 arranged tointercept and detect reflections of the light pulses or beams emitted bytransmitter 120 that return to system 100 from the environment. Examplelight detectors 132 may include photodiodes, avalanche photodiodes(APDs), silicon photomultipliers (SiPMs), single photon avalanche diodes(SPADs), multi-pixel photon counters (MPPCs), phototransistors, cameras,active pixel sensors (APS), charge coupled devices (CCD), cryogenicdetectors, and/or any other sensor of light. In some instances, receiver130 may be configured to detect light having wavelengths in the samewavelength range as the light emitted by transmitter 120. In this way,for instance, system 100 may distinguish received light originated bysystem 100 from other light originated by external sources in theenvironment.

In some implementations, receiver 130 may include a detector comprisingan array of sensing elements connected to one another. For instance,where system 100 is configured as a LIDAR device, multiple light sensingelements could be connected in parallel to provide a photodetector arrayhaving a larger light detection area (e.g., combination of the sensingsurfaces of the individual detectors in the array, etc.) than adetection area of a single sensing element. The photodetector arraycould be arranged in a variety ways. For instance, the individualdetectors of the array can be disposed on one or more substrates (e.g.,printed circuit boards (PCBs), flexible PCBs, etc.) and arranged todetect incoming light that is traveling along an optical path of anoptical lens of system 100 (e.g., optical element(s) 140). Also, such aphotodetector array could include any feasible number of detectorsarranged in any feasible manner.

In some examples, system 100 can select or adjust a horizontal scanningresolution by changing a rate of rotation of system 100 (and/ortransmitter 120 and receiver 130). Additionally or alternatively, thehorizontal scanning resolution can be modified by adjusting a pulse rateof signals emitted by transmitter 120. In a first example, transmitter120 may be configured to emit pulses at a pulse rate of 15,650 pulsesper second, and to rotate at 10 Hz (i.e., ten complete 360° rotationsper second) while emitting the pulses. In this example, receiver 130 mayhave a 0.23° horizontal angular resolution (e.g., horizontal angularseparation between consecutive pulses). In a second example, if system100 is instead rotated at 20 Hz while maintaining the pulse rate of15,650 pulses per second, then the horizontal angular resolution maybecome 0.46°. In a third example, if transmitter 120 emits the pulses ata rate of 31,300 pulses per second while maintaining the rate ofrotation of 10 Hz, then the horizontal angular resolution may become0.115°. In some examples, system 100 can be alternatively configured toscan a particular range of views within less than a complete 360°rotation of system 100. Other implementations are possible as well.

It is noted that the pulse rates, angular resolutions, rates ofrotation, and viewing ranges described above are only for the sake ofexample, and thus each of these scanning characteristics could varyaccording to various applications of system 100.

Optical element(s) 140 can be optionally included in or otherwisecoupled to transmitter 120 and/or receiver 130. In one example, opticalelement(s) 140 can be arranged to direct light emitted by emitter(s) 122toward a scene (or a region therein). In another example, opticalelement(s) 140 can be arranged to focus light from the scene (or aregion therein) toward detector(s) 132. As such, optical element(s) 140may include any feasible combination of optical elements, such asfilters, apertures, mirror(s), waveguide(s), lens(es), or other typesoptical components, that are arranged to guide propagation of lightthrough physical space and/or to adjust a characteristic of the light.

In some examples, controller 104 could operate actuator 114 to rotaterotating platform 110 in various ways so as to obtain information aboutthe environment. In one example, rotating platform 110 could be rotatedin either direction. In another example, rotating platform 110 may carryout complete revolutions such that system 100 scans a 360° view of theenvironment. Moreover, rotating platform 110 could rotate at variousfrequencies so as to cause system 100 to scan the environment at variousrefresh rates. In one embodiment, system 100 may be configured to have arefresh rate of 10 Hz (e.g., ten complete rotations of system 100 persecond). Other refresh rates are possible.

Alternatively or additionally, system 100 may be configured to adjustthe pointing direction of an emitted signal (emitted by transmitter 120)in various ways. In one implementation, signal emitters (e.g., lightsources, antennas, acoustic transducers, etc.) of transmitter 120 can beoperated according to a phased array configuration or other type of beamsteering configuration.

In a first example, where system 100 is configured as a LIDAR device,light sources or emitters in transmitter 120 can be coupled to phasedarray optics that control the phase of light waves emitted by the lightsources. For instance, controller 104 can be configured to adjust thephased array optics (e.g., phased array beam steering) to change theeffective pointing direction of a light signal emitted by transmitter120 (e.g., even if rotating platform 110 is not rotating).

In a second example, where system 100 is configured as a RADAR device,transmitter 120 may include an array of antennas, and controller 104 canprovide respective phase-shifted control signals for each individualantenna in the array to modify a pointing direction of a combined RFsignal from the array (e.g., phased array beam steering).

In a third example, where system 100 is configured as a SONAR device,transmitter 120 may include an array of acoustic transducers, andcontroller 104 can similarly operate the array of acoustic transducers(e.g., via phase-shifted control signals, phased array beam steering,etc.) to achieve a target pointing direction of a combined sound signalemitted by the array (e.g., even if rotating platform 110 is notrotating, etc.).

Housing 150 may take on any shape, form, and material and may beconfigured to house one or more components of system 100. In oneexample, housing 150 can be a dome-shaped housing. Further, in someexamples, housing 150 may be composed of or may include a material thatis at least partially non-transparent, which may allow for blocking ofat least some signals from entering the interior space of the housing150 and thus help mitigate thermal and noise effects of ambient signalson one or more components of system 100. Other configurations of housing150 are possible as well.

In some examples, housing 150 may be coupled to rotating platform 110such that housing 150 is configured to rotate based on rotation ofrotating platform 110. In these examples, transmitter 120, receiver 130,and possibly other components of system 100 may each be disposed withinhousing 150. In this manner, transmitter 120 and receiver 130 may rotatealong with housing 150 while being disposed within housing 150. In otherexamples, housing 150 may be coupled to stationary platform 112 or otherstructure such that housing 150 does not rotate with the othercomponents rotated by rotating platform 110.

As shown, housing 150 can optionally include a first optical window 152and a second optical window 154. Thus, in some examples, housing 150 maydefine an optical cavity in which one or more components disposed insidethe housing (e.g., transmitter 120, receiver 130, etc.) are opticallyisolated from external light in the environment, except for light thatpropagates through optical windows 152 and 154. With this arrangementfor instance, system 100 (e.g., in a LIDAR configuration, etc.) mayreduce interference from external light (e.g., noise, etc.) with signalstransmitted by transmitter 120 and/or reflections of the transmittedsignal received by receiver 130.

To that end, in some embodiments, optical windows 152 and 154 mayinclude a material that is transparent to the wavelengths of lightemitted by emitters 122 and/or one or more other wavelengths. Forexample, each of optical windows 152 and 154 may be formed from a glasssubstrate or a plastic substrate, among others. Additionally, in someexamples, each of optical windows 152 and 154 may include or may becoupled to a filter that selectively transmits wavelengths of lighttransmitted by emitter(s) 122, while reducing transmission of otherwavelengths. Optical windows 152 and 154 may have various thicknesses.In one embodiment, optical windows 152 and 154 may have a thicknessbetween 1 millimeter and 2 millimeters. Other thicknesses are possibleas well.

In some examples, second optical window 154 may be located at anopposite side of housing 150 from first optical window 152.

Cleaning apparatus(es) 160 can be optionally included in system 100 tofacilitate cleaning one or more components (e.g., optical element(s)140, etc.) of system 100. To that end, cleaning apparatus 160 mayinclude one or more cleaning mechanisms. A first example cleaningapparatus 160 may include a liquid spray configured to deposit liquid onone or more components of system 100 (e.g., optical element(s) 140,housing 150, etc.). For instance, the liquid can be applied to attemptdissolving or mechanically removing an occlusion (e.g., dirt, dust,etc.) disposed on a surface of an optical component. A second examplecleaning apparatus 160 may include a high-pressure gas pump configuredto apply gas onto an occlusion on a surface of an optical component. Athird example cleaning apparatus 160 may include a wiper (e.g., similarto a windshield wiper) configured to attempt removing an occlusion froma surface of a component in system 100. Other examples are possible.

It is noted that this arrangement of system 100 is described forexemplary purposes only and is not meant to be limiting. As noted above,in some examples, system 100 can be alternatively implemented with fewercomponents than those shown. In one example, system 100 can beimplemented without rotating platform 100. For instance, transmitter 120can be configured to transmit a plurality of signals spatially arrangedto define a particular FOV of system 100 (e.g., horizontally andvertically) without necessarily rotating transmitter 120 and receiver130. In another example, housing 150 may be configured to include asingle optical window (instead of two optical windows 152, 154). Forinstance, in a LIDAR implementation of system 100, LIDAR 100 may obtaina single complete scan of a FOV during a given scan period bytransmitting light pulses through the single optical window only (i.e.,instead of obtaining two scans of the same FOV simultaneously bytransmitting light pulses through two separate optical windows). Otherexamples are possible as well.

FIG. 2A illustrates a LIDAR device 200, according to exampleembodiments. As shown, LIDAR 200 includes a rotating platform 210, astationary platform 212, and a housing 250 that are similar,respectively, to rotating platform 110, stationary platform 112, andhousing 150 of system 100.

LIDAR 200 may be configured to scan an environment by emitting light 260toward the environment, and detecting reflect portions (e.g., reflectedlight 270) of the emitted light returning to LIDAR 200 from theenvironment. Further, to adjust a FOV scanned by LIDAR 200 (i.e., theregion illuminated by emitted light 260), rotating platform 210 may beconfigured to rotate housing 250 (and one or more components includedtherein) about an axis of rotation of rotating platform 210. Forinstance, where the axis of rotation of platform 210 is a vertical axis,rotating platform 210 may adjust the direction of emitted light 260horizontally to define the horizontal extents of the FOV of LIDAR 200.

As shown, LIDAR 200 also includes an optical window 252 through whichemitted light 260 is transmitted out of housing 250, and through whichreflected light 270 enters into housing 250. Although not shown, housing250 may also include another optical window located at an opposite sideof housing 250 from optical window 252. Thus, housing 250 may define anoptical cavity in which one or more components disposed inside thehousing (e.g., transmitter, receiver, etc.) are optically isolated fromexternal light in the environment, except for light that propagatesthrough one or more optical windows. With this arrangement for instance,LIDAR 200 may reduce interference from external light (e.g., noise,etc.) with transmitted signals 260 and/or reflected signals 270.

To that end, in some embodiments, optical window 252 may include amaterial that is transparent to the wavelengths of emitted light 270and/or one or more other wavelengths. For example, optical window 252may be formed from a glass substrate or a plastic substrate, amongothers. Additionally, in some examples, optical window 252 may includeor may be coupled to a filter that selectively transmits wavelengths ofemitted light 260, while reducing transmission of other wavelengthsthrough the optical window 252. Optical window 252 may have variousthicknesses. In one embodiment, optical window 252 may have a thicknessbetween 1 millimeter and 2 millimeters. Other thicknesses are possible.

FIG. 2B illustrates a partial cross-section view of LIDAR 200. It isnoted that some of the components of LIDAR 200 (e.g., platform 212,housing 250, and optical window 252) are omitted from the illustrationof FIG. 2B for convenience in description.

As shown in FIG. 2B, LIDAR device 200 also includes actuators 214 and218, which may be similar to actuators 114 of system 100. Additionally,as shown, LIDAR 200 includes a transmitter 220 and a receiver 230, whichmay be similar, respectively, to transmitter 120 and receiver 130 ofsystem 100. Additionally, as shown, LIDAR 200 includes one or moreoptical elements (i.e., a transmit lens 240, a receive lens 242, and amirror 244), which may be similar to optical elements 140 of system 100.

Actuators 214 and 218 may include a stepper motor, an electric motor, acombustion motor, a pancake motor, a piezoelectric actuator, or anyother type of actuator, such as those describe for actuators 114 ofsystem 100.

As shown, actuator 214 may be configured to rotate the mirror 244 abouta first axis 215, and actuator 218 may be configured to rotate rotatingplatform 210 about a second axis 219. In some embodiments, axis 215 maycorrespond to a horizontal axis of LIDAR 200 and axis 219 may correspondto a vertical axis of LIDAR 200 (e.g., axes 215 and 219 may be alignedsubstantially perpendicular to one another).

In an example operation, LIDAR transmitter 220 may emit light (viatransmit lens 240) that reflects off mirror 244 to propagate away fromLIDAR 200 (e.g., as emitted light 260 shown in FIG. 2A). Further,received light from the environment of LIDAR 200 (including light 270shown in FIG. 2A) may be reflected off mirror 244 toward LIDAR receiver230 (via lens 242). Thus, for instance, a vertical scanning direction ofLIDAR 200 can be controlled by rotating mirror 244 (e.g., about ahorizontal axis 215), and a horizontal scanning direction of LIDAR 200can be controlled by rotating LIDAR 200 about a vertical axis (e.g.,axis 219) using rotating platform 210.

In this example, mirror 244 could be rotated while transmitter 220 isemitting a series of light pulses toward the mirror. Thus, depending onthe rotational position of the mirror about axis 215, each light pulsecould thus be steered (e.g., vertically). As such, LIDAR 200 may scan avertical FOV defined by a range of (vertical) steering directionsprovided by mirror 244 (e.g., based on a range of angular positions ofmirror 244 about axis 215). In some examples, LIDAR 200 may beconfigured to rotate mirror 244 one or more complete rotations to steeremitted light from transmitter 220 (vertically). In other examples,LIDAR device 200 may be configured to rotate mirror 244 within a givenrange of angles to steer the emitted light over a particular range ofdirections (vertically). Thus, LIDAR 200 may scan a variety of verticalFOVs by adjusting the rotation of mirror 244. In one embodiment, thevertical FOV of LIDAR 200 is 110°.

Continuing with this example, platform 210 may be configured to rotatethe arrangement of components supported thereon (e.g., mirror 244, motor214, lenses 230 and 232, transmitter 220, and receiver 230) about avertical axis (e.g., axis 219). Thus, LIDAR 200 may rotate platform 210to steer emitted light (from transmitter 220) horizontally (e.g., aboutthe axis of rotation 219 of platform 210). Additionally, the range ofthe rotational positions of platform 210 (about axis 219) can becontrolled to define a horizontal FOV of LIDAR 200. In one embodiment,platform 210 may rotate within a defined range of angles (e.g., 270°,etc.) to provide a horizontal FOV that is less than 360°. However, otheramounts of rotation are possible as well (e.g., 360°, 8°, etc.) to scanany horizontal FOV.

FIG. 2C illustrates a partial cross-section view of LIDAR device 200. Itis noted that some of the components of LIDAR 200 are omitted from theillustration of FIG. 2C for convenience in description. In thecross-section view of FIG. 2C, axis 215 may be perpendicular to (and mayextend through) the page.

As shown in FIG. 2C, LIDAR 200 also includes a second optical window 254that is positioned opposite to optical window 252. Optical window 254may be similar to optical window 252. For example, optical window 254may be configured to transmit light into and/or out of the opticalcavity defined by housing 250.

As shown in FIG. 2C, transmitter 220 includes an emitter 222, which mayinclude any of the light sources described for emitter(s) 122, forinstance. In alternative embodiments, transmitter 220 may include morethan one light source. Emitter 222 may be configured to emit one or morelight pulses 260 (e.g., laser beams, etc.). Transmit lens 240 may beconfigured to direct (and/or collimate) the emitted light from emitter222 toward mirror 244. For example, transmit lens 240 may collimate thelight from the emitter to define a beam width of the light beam 260transmitted out of LIDAR 200 (e.g., the beam divergence angle betweendotted lines 260 a and 260 b).

As shown in FIG. 2C, mirror 244 may include three reflective surfaces244 a, 244 b, 244 c (e.g., triangular mirror). In alternative examples,mirror 244 may instead include additional or fewer reflective surfaces.In the example shown, the emitted light transmitted through transmitlens 240 may then reflect off reflective surface 244 a toward theenvironment of LIDAR 200 in the direction illustrated by arrow 260.Thus, in this example, as mirror 244 is rotated (e.g., about axis 215),emitted light 260 may be steered to have a different direction (e.g.,pitch direction, etc.) than that illustrated by arrow 260. For example,the direction 260 of the emitted light could be adjusted based on therotational position of triangular mirror 244.

Additionally, in some examples, emitted light 260 may be steered out ofhousing 250 through optical window 252 or through optical window 254depending on the rotational position of mirror 244 about axis 215. Thus,in some examples, LIDAR 200 may be configured to steer emitted lightbeam 260 within a wide range of directions (e.g., vertically), and/orout of either side of housing 250 (e.g., the sides where optical windows252 and 252 are located).

FIG. 2D illustrates another partial cross-section view of LIDAR device200. It is noted that some of the components of LIDAR 200 are omittedfrom the illustration of FIG. 2D for convenience in description. Asshown, receiver 230 includes one or more light detectors 232, which maybe similar to detector(s) 112 of system 100. Further, as shown, receiver230 includes a diaphragm 246 between receive lens 246 and detector(s)232.

Diaphragm 246 may include one or more optical elements (e.g., aperturestop, filter, etc.) configured to select a portion the light focused byreceive lens 242 for transmission toward detector(s) 232.

For example, receive lens 242 may be configured to focus light receivedfrom the scene scanned by LIDAR 200 (e.g., light from the scene thatenters window 252 or window 254 and is reflected by mirror 244) towarddiaphragm 246. In line with the discussion above, detector(s) 232 may bearranged (or aligned) to intercept a portion of the focused light thatincludes light from the target region illuminated by transmitter 220. Tofacilitate this, for example, diaphragm 246 may include an aperturepositioned and/or sized to transmit the portion of the focused lightassociated with the target region through the aperture as diverginglight (e.g., including reflected light 270) for detection by detector(s)232.

It is noted that the various positions, shapes, and sizes of the variouscomponents of LIDAR 200, the light beams emitted by LIDAR 200, and/orthe light beams received by LIDAR 200 may vary and are not necessarilyto scale, but are illustrated as shown in FIGS. 2A-2D for convenience indescription. Additionally, it is noted that LIDAR 200 may alternativelyinclude additional, fewer, or different components than those shown inFIGS. 2A-2D.

In one alternative implementation, housing 250 may be configured toinclude a single optical window (instead of two optical windows 252 and254). For example, LIDAR 200 may be configured to obtain a singlecomplete scan of the FOV of LIDAR 200 during each rotation of thehousing 250 across the range of yaw angles associated with thehorizontal extents of the FOV of LIDAR 200. In this example, LIDAR 200may be configured to transmit light pulses through the single opticalwindow during the rotation of housing 250 to obtain a single scan of theFOV during a given scan period (i.e., instead of obtaining two scans ofthe same FOV by transmitting light pulses through two separate opticalwindows simultaneously over the given scan period).

In another alternative implementation, LIDAR 200 may include abeam-steering apparatus comprising one or more optical componentsinstead of or in addition to mirror 244. In this implementation, LIDAR200 may be configured to scan the scene by steering the transmittedand/or received light beams through optical windows 252 and/or 254 usingvarious arrangements of beam-steering optical elements.

Some example implementations herein involve a vehicle includes at leastone sensor, such as system 100, device 200, and/or other types ofsensors (e.g., RADARs, SONARs, cameras, other active sensors, etc.).However, an example sensor disclosed herein can also be used for variousother purposes and may be incorporated in or otherwise connected to anyfeasible system or arrangement. For instance, an example LIDAR deviceherein can be used in an assembly line setting to monitor objects (e.g.,products) being manufactured in the assembly line. Other examples arepossible as well.

Some example LIDAR devices herein may be used with any type of vehicle,including conventional automobiles as well as automobiles having anautonomous or semi-autonomous mode of operation. Further, the term“vehicle” is to be broadly construed to cover any moving object,including, for instance, a truck, a van, a semi-trailer truck, amotorcycle, a golf cart, an off-road vehicle, a warehouse transportvehicle, or a farm vehicle, as well as a carrier that rides on a tracksuch as a rollercoaster, trolley, tram, or train car, etc.

FIG. 3 is a simplified block diagram of a vehicle 300, according to anexample embodiment. As shown, the vehicle 300 includes a propulsionsystem 302, a sensor system 304, a control system 306, peripherals 308,and a computer system 310. In some embodiments, vehicle 300 may includemore, fewer, or different systems, and each system may include more,fewer, or different components. Additionally, the systems and componentsshown may be combined or divided in any number of ways. For instance,control system 306 and computer system 310 may be combined into a singlesystem.

Propulsion system 302 may be configured to provide powered motion forthe vehicle 300. To that end, as shown, propulsion system 302 includesan engine/motor 318, an energy source 320, a transmission 322, andwheels/tires 324.

The engine/motor 318 may be or include any combination of an internalcombustion engine, an electric motor, a steam engine, and a Sterlingengine. Other motors and engines are possible as well. In someembodiments, propulsion system 302 may include multiple types of enginesand/or motors. For instance, a gas-electric hybrid car may include agasoline engine and an electric motor. Other examples are possible.

Energy source 320 may be a source of energy that powers the engine/motor318 in full or in part. That is, engine/motor 318 may be configured toconvert energy source 320 into mechanical energy. Examples of energysources 320 include gasoline, diesel, propane, other compressedgas-based fuels, ethanol, solar panels, batteries, and other sources ofelectrical power. Energy source(s) 320 may additionally or alternativelyinclude any combination of fuel tanks, batteries, capacitors, and/orflywheels. In some embodiments, energy source 320 may provide energy forother systems of the vehicle 300 as well. To that end, energy source 320may additionally or alternatively include, for example, a rechargeablelithium-ion or lead-acid battery. In some embodiments, energy source 320may include one or more banks of batteries configured to provide theelectrical power to the various components of vehicle 300.

Transmission 322 may be configured to transmit mechanical power from theengine/motor 318 to the wheels/tires 324. To that end, transmission 322may include a gearbox, clutch, differential, drive shafts, and/or otherelements. In embodiments where the transmission 322 includes driveshafts, the drive shafts may include one or more axles that areconfigured to be coupled to the wheels/tires 324.

Wheels/tires 324 of vehicle 300 may be configured in various formats,including a unicycle, bicycle/motorcycle, tricycle, or car/truckfour-wheel format. Other wheel/tire formats are possible as well, suchas those including six or more wheels. In any case, wheels/tires 324 maybe configured to rotate differentially with respect to otherwheels/tires 324. In some embodiments, wheels/tires 324 may include atleast one wheel that is fixedly attached to the transmission 322 and atleast one tire coupled to a rim of the wheel that could make contactwith the driving surface. Wheels/tires 324 may include any combinationof metal and rubber, or combination of other materials. Propulsionsystem 302 may additionally or alternatively include components otherthan those shown.

Sensor system 304 may include a number of sensors configured to senseinformation about an environment in which the vehicle 300 is located, aswell as one or more actuators 336 configured to modify a position and/ororientation of the sensors. As shown, sensor system 304 includes aGlobal Positioning System (GPS) 326, an inertial measurement unit (IMU)328, a RADAR unit 330, a laser rangefinder and/or LIDAR unit 332, and acamera 334. Sensor system 304 may include additional sensors as well,including, for example, sensors that monitor internal systems of thevehicle 300 (e.g., an O₂ monitor, a fuel gauge, an engine oiltemperature, etc.). Other sensors are possible as well.

GPS 326 may be any sensor (e.g., location sensor) configured to estimatea geographic location of vehicle 300. To this end, the GPS 326 mayinclude a transceiver configured to estimate a position of the vehicle300 with respect to the Earth.

IMU 328 may be any combination of sensors configured to sense positionand orientation changes of the vehicle 300 based on inertialacceleration. In some embodiments, the combination of sensors mayinclude, for example, accelerometers, gyroscopes, compasses, etc.

RADAR unit 330 may be any sensor configured to sense objects in theenvironment in which the vehicle 300 is located using radio signals. Insome embodiments, in addition to sensing the objects, RADAR unit 330 mayadditionally be configured to sense the speed and/or heading of theobjects.

Similarly, laser range finder or LIDAR unit 332 may be any sensorconfigured to sense objects in the environment in which vehicle 300 islocated using lasers. For example, LIDAR unit 332 may include one ormore LIDAR devices, which may be similar to system 100 and/or device 200among other possible LIDAR configurations.

Camera 334 may be any camera (e.g., a still camera, a video camera,etc.) configured to capture images of the environment in which thevehicle 300 is located. To that end, camera 334 may take any of theforms described above.

Control system 306 may be configured to control one or more operationsof vehicle 300 and/or components thereof. To that end, control system306 may include a steering unit 338, a throttle 340, a brake unit 342, asensor fusion algorithm 344, a computer vision system 346, navigation orpathing system 348, and an obstacle avoidance system 350.

Steering unit 338 may be any combination of mechanisms configured toadjust the heading of vehicle 300. Throttle 340 may be any combinationof mechanisms configured to control engine/motor 318 and, in turn, thespeed of vehicle 300. Brake unit 342 may be any combination ofmechanisms configured to decelerate vehicle 300. For example, brake unit342 may use friction to slow wheels/tires 324. As another example, brakeunit 342 may convert kinetic energy of wheels/tires 324 to an electriccurrent.

Sensor fusion algorithm 344 may be an algorithm (or a computer programproduct storing an algorithm) configured to accept data from sensorsystem 304 as an input. The data may include, for example, datarepresenting information sensed by sensor system 304. Sensor fusionalgorithm 344 may include, for example, a Kalman filter, a Bayesiannetwork, a machine learning algorithm, an algorithm for some of thefunctions of the methods herein, or any other sensor fusion algorithm.Sensor fusion algorithm 344 may further be configured to provide variousassessments based on the data from sensor system 304, including, forexample, evaluations of individual objects and/or features in theenvironment in which vehicle 300 is located, evaluations of particularsituations, and/or evaluations of possible impacts based on particularsituations. Other assessments are possible as well.

Computer vision system 346 may be any system configured to process andanalyze images captured by camera 334 in order to identify objectsand/or features in the environment in which vehicle 300 is located,including, for example, traffic signals and obstacles. To that end,computer vision system 346 may use an object recognition algorithm, aStructure from Motion (SFM) algorithm, video tracking, or other computervision techniques. In some embodiments, computer vision system 346 mayadditionally be configured to map the environment, track objects,estimate the speed of objects, etc.

Navigation and pathing system 348 may be any system configured todetermine a driving path for vehicle 300. Navigation and pathing system348 may additionally be configured to update a driving path of vehicle300 dynamically while vehicle 300 is in operation. In some embodiments,navigation and pathing system 348 may be configured to incorporate datafrom sensor fusion algorithm 344, GPS 326, LIDAR unit 332, and/or one ormore predetermined maps so as to determine a driving path for vehicle300.

Obstacle avoidance system 350 may be any system configured to identify,evaluate, and avoid or otherwise negotiate obstacles in the environmentin which vehicle 300 is located. Control system 306 may additionally oralternatively include components other than those shown.

Peripherals 308 may be configured to allow vehicle 300 to interact withexternal sensors, other vehicles, external computing devices, and/or auser. To that end, peripherals 308 may include, for example, a wirelesscommunication system 352, a touchscreen 354, a microphone 356, and/or aspeaker 358.

Wireless communication system 352 may be any system configured towirelessly couple to one or more other vehicles, sensors, or otherentities, either directly or via a communication network. To that end,wireless communication system 352 may include an antenna and a chipsetfor communicating with the other vehicles, sensors, servers, or otherentities either directly or via a communication network. The chipset orwireless communication system 352 in general may be arranged tocommunicate according to one or more types of wireless communication(e.g., protocols) such as Bluetooth, communication protocols describedin IEEE 802.11 (including any IEEE 802.11 revisions), cellulartechnology (such as GSM, CDMA, UMTS, EV-DO, WiMAX, or LTE), Zigbee,dedicated short range communications (DSRC), and radio frequencyidentification (RFID) communications, among other possibilities.

Touchscreen 354 may be used by a user to input commands to vehicle 300.To that end, touchscreen 354 may be configured to sense at least one ofa position and a movement of a user's finger via capacitive sensing,resistance sensing, or a surface acoustic wave process, among otherpossibilities. Touchscreen 354 may be capable of sensing finger movementin a direction parallel or planar to the touchscreen surface, in adirection normal to the touchscreen surface, or both, and may also becapable of sensing a level of pressure applied to the touchscreensurface. Touchscreen 354 may be formed of one or more translucent ortransparent insulating layers and one or more translucent or transparentconducting layers. Touchscreen 354 may take other forms as well.

Microphone 356 may be configured to receive audio (e.g., a voice commandor other audio input) from a user of vehicle 300. Similarly, speakers358 may be configured to output audio to the user.

Computer system 310 may be configured to transmit data to, receive datafrom, interact with, and/or control one or more of propulsion system302, sensor system 304, control system 306, and peripherals 308. To thisend, computer system 310 may be communicatively linked to one or more ofpropulsion system 302, sensor system 304, control system 306, andperipherals 308 by a system bus, network, and/or other connectionmechanism (not shown).

In one example, computer system 310 may be configured to controloperation of transmission 322 to improve fuel efficiency. As anotherexample, computer system 310 may be configured to cause camera 334 tocapture images of the environment. As yet another example, computersystem 310 may be configured to store and execute instructionscorresponding to sensor fusion algorithm 344. As still another example,computer system 310 may be configured to store and execute instructionsfor determining a 3D representation of the environment around vehicle300 using LIDAR unit 332. Thus, for instance, computer system 310 couldfunction as a controller for LIDAR unit 332. Other examples are possibleas well.

As shown, computer system 310 includes processor 312 and data storage314. Processor 312 may comprise one or more general-purpose processorsand/or one or more special-purpose processors. To the extent thatprocessor 312 includes more than one processor, such processors couldwork separately or in combination.

Data storage 314, in turn, may comprise one or more volatile and/or oneor more non-volatile storage components, such as optical, magnetic,and/or organic storage, and data storage 314 may be integrated in wholeor in part with processor 312. In some embodiments, data storage 314 maycontain instructions 316 (e.g., program logic) executable by processor312 to cause vehicle 300 and/or components thereof (e.g., LIDAR unit332, etc.) to perform the various operations described herein. Datastorage 314 may contain additional instructions as well, includinginstructions to transmit data to, receive data from, interact with,and/or control one or more of propulsion system 302, sensor system 304,control system 306, and/or peripherals 308.

In some embodiments, vehicle 300 may include one or more elements inaddition to or instead of those shown. For example, vehicle 300 mayinclude one or more additional interfaces and/or power supplies. Otheradditional components are possible as well. In such embodiments, datastorage 314 may also include instructions executable by processor 312 tocontrol and/or communicate with the additional components. Stillfurther, while each of the components and systems are shown to beintegrated in vehicle 300, in some embodiments, one or more componentsor systems may be removably mounted on or otherwise connected(mechanically or electrically) to vehicle 300 using wired or wirelessconnections. Vehicle 300 may take other forms as well.

III. EXAMPLE METHODS

It is noted that the example arrangement described for LIDAR 200 is notmeant to be limiting. Thus, the methods and processes described hereincan be used with a variety of different LIDAR configurations, includingLIDAR device 200 as well as other LIDAR arrangements. Additionally, themethods and processes described herein can be used with a variety ofdifferent types of active sensors such as any of the active sensingsystems in the description of system 100 (e.g., SONARs, RADARs, LIDARs,etc.).

FIG. 4 is a flowchart of a method 400, according to example embodiments.Method 400 presents an embodiment of a method that could be used withany of system 100, device 200, and/or vehicle 300, for example. Method400 may include one or more operations, functions, or actions asillustrated by one or more of blocks 402-408. Although the blocks areillustrated in a sequential order, these blocks may in some instances beperformed in parallel, and/or in a different order than those describedherein. Also, the various blocks may be combined into fewer blocks,divided into additional blocks, and/or removed based upon the desiredimplementation.

In addition, for method 400 and other processes and methods disclosedherein, the flowchart shows functionality and operation of one possibleimplementation of present embodiments. In this regard, each block mayrepresent a module, a segment, a portion of a manufacturing or operationprocess, or a portion of program code, which includes one or moreinstructions executable by a processor for implementing specific logicalfunctions or steps in the process. The program code may be stored on anytype of computer readable medium, for example, such as a storage deviceincluding a disk or hard drive. The computer readable medium may includea non-transitory computer readable medium, for example, such ascomputer-readable media that stores data for short periods of time likeregister memory, processor cache and Random Access Memory (RAM). Thecomputer readable medium may also include non-transitory media, such assecondary or persistent long term storage, like read only memory (ROM),optical or magnetic disks, compact-disc read only memory (CD-ROM), forexample. The computer readable media may also be any other volatile ornon-volatile storage systems. The computer readable medium may beconsidered a computer readable storage medium, for example, or atangible storage device.

In addition, for method 400 and other processes and methods disclosedherein, each block in FIG. 4 may represent circuitry that is wired toperform the specific logical functions in the process.

At block 402, method 400 involves rotating a housing of a LIDAR deviceabout a first axis. As shown in FIG. 2B for example, LIDAR device 200may rotate housing 250 about axis 219 (e.g., by using rotating platform210 which supports housing 250, etc.) to adjust a horizontal pointingdirection (e.g., yaw direction) of emitted light 260 (shown in FIG. 2A).

At block 404, method 400 involves transmitting a first plurality oflight pulses through a first optical window of the housing to obtain afirst scan of a FOV.

As shown in FIG. 2C for example, LIDAR 200 may transmit (at least aportion of) emitted light 260 through first optical window 252 ofhousing 250. In some examples, emitter 222 of LIDAR 200 may emit light260 as a series of light pulses (i.e., the first plurality of lightpulses at block 404). For instance, emitted light pulses 260 can beemitted periodically, intermittently, or may be separated temporallyaccording to another modulation scheme. Additionally, as rotatingplatform 210 rotates about axis 219 (shown in FIG. 2B), each emittedlight pulse may be steered horizontally to have a respective yawdirection according to a current rotational position of platform 210about axis 219. Thus, for instance, the first plurality of light pulsesat block 404 may be transmitted through first optical window 252 over arange of yaw angles (i.e., range of rotational positions of platform 210about axis 219) to obtain the first scan of a horizontal FOV of LIDAR200.

In some examples, LIDAR 200 may be configured to repeatedly scan a FOVthat extends horizontally over 360 degrees via the first optical window252. For instance, actuator 218 (shown in FIG. 2B) may be configured torepeatedly rotate platform 210 complete rotations (i.e., 360 degreerotations) about axis 219. As such, LIDAR 200 may be configured toobtain a full scan (e.g., of the 360-degree FOV) through the firstoptical window 252 during each complete rotation of platform 210 aboutaxis 219.

At block 406, method 400 involves transmitting a second plurality oflight pulses through a second window of the housing.

As shown in FIG. 2C for example, housing 250 includes second opticalwindow 254 (opposite first optical window 252). In an example scenario,mirror 244 could be rotated in a clockwise direction about axis 215(relative to the configuration of mirror 244 shown in FIG. 2C). In thisscenario, reflective surface 244 b may begin to intercept and reflectemitted light 260 from transmitter 220 out of the second optical window254. More generally, in this example, the second plurality of lightpulses at block 406 may include a subset of emitted light pulses 260that are emitted by emitter 222 during times when any of reflectivesurfaces 244 a, 244 b, or 244 c is facing both transmitter 220 andoptical window 254. Similarly, the first plurality of light pulses atblock 404 may include a subset of emitted light pulses 260 that areemitted by emitter 222 during times when any of reflective surfaces 244a, 244 b, or 244 c is facing both transmitter 220 and optical window252.

As noted above, in some examples, LIDAR 200 may be configured torepeatedly scan a FOV via the first optical window 252 and(simultaneously) scan the same FOV via the second optical window 254.For example, LIDAR 200 may be configured to obtain two separate scans(e.g., the first scan of block 404 and the second scan of block 406) ofthe same FOV during a same scanning time period by interchangeablytransmitting light beams out of optical windows 252 and 254 (e.g., basedon the rotation of mirror 244).

In some examples, method 400 involves obtaining the first scan of theFOV and the second scan of the FOV based on measurements of reflectedlight pulses intercepted by one or more detectors of the LIDAR deviceduring one complete rotation of the housing about the first axis.Referring back to FIG. 2D for example, detector(s) 232 can be used todetect reflected light pulses 270, which are received by LIDAR 200 viaany of optical windows 252 and 254 (and reflected by mirror 244 towardreceiver 230) during one complete (360 degree) rotation of housing 250about axis 219 (shown in FIG. 2B). In this way, during each completerotation of housing 250 about axis 219 (e.g., 360 degree rotation),LIDAR 200 can obtain two separate scans (the first scan via window 252and the second scan via window 254) of the same FOV.

In some examples, method 400 involves emitting, by a transmitter insidethe housing, the first plurality of light pulses and the secondplurality of light pulses toward a mirror inside the housing. Forexample, as shown in FIG. 2C, the same transmitter 220 may emit both thefirst plurality of light pulses (i.e., portion of emitted light 260reflected by mirror 244 toward optical window 252) and the secondplurality of light pulses (i.e., another portion of emitted light 260reflected by mirror 244 toward optical window 254).

Thus, in some examples, method 400 may also involve the transmittertransmitting the first plurality of light pulses and the secondplurality of light pulses toward a mirror inside the housing; andreflecting, via the mirror, the first plurality of light pulses towardthe first optical window and the second plurality of light pulses towardthe second optical window.

Referring back to FIG. 2C, LIDAR 200 may use the same optical components(e.g., emitter 222, transmit lens 240, mirror 244, etc.) to define therespective optical paths of the first plurality of light pulses(transmitted through window 252) and the second plurality of lightpulses (transmitted through window 254). In this way, the two scans(e.g., the first scan and the second scan) obtained through both opticalwindows 252 and 254 could be expected to be substantially similar to oneanother (e.g., temporally and spatially).

By way of example, consider a scenario where platform 210 rotates at arate of 10 Hertz (Hz). In this scenario, the first scan and the secondscan of the FOV may be performed during a same scanning period of 0.1seconds (s). For instance, at time t=0, LIDAR 200 may transmit avertical column of light pulses at a yaw angle of 0° (through opticalwindow 252) and then a vertical column of light pulses at a yaw angle of180° (through optical window 254). Between t=0 and t=0.05 s (i.e., firsthalf the scanning period), platform 210 in this scenario may rotate for180° about yaw axis 219 while similarly transmitting vertical columns oflight pulses out of both windows 252 and 254 interchangeably. Further,at t=0.05 s, LIDAR 200 may then transmit another vertical column oflight pulses at the yaw angle of 0° (through optical window 254) andanother vertical column of light pulses at the yaw angle of 180°(through optical window 252). Next, between time t=0.05 s and t=0.1 s(i.e., second half of the scanning period), LIDAR 200 may then scan theyaw angles between 0° and 180° via window 254 and the yaw angles between180° and 360° via window 252.

Thus, in this scenario, the first scan and the second may besubstantially similar (e.g., approximately 0.05 seconds delay betweencorresponding light pulses of the two scans that are transmitted in asubstantially similar direction). Other scenarios are possible.

At block 408, method 400 involves identifying a portion of the FOV thatis at least partially occluded by an occlusion. For instance, theocclusion may intercept at least a portion of light transmitted by theLIDAR device toward the identified portion of the FOV.

The identification of the portion of the FOV at block 408 could be basedon the first scan, the second scan, or both the first scan and thesecond scan. Thus, in some examples, a system of method 400 may beconfigured to identify the portion of the FOV at block 408 based on oneor more scans of the FOV obtained by the LIDAR device via a singleoptical window. In other examples, a system of method 400 may beconfigured to identify the portion of the FOV at block 408 based on oneor more scans of the FOV obtained by the LIDAR device via both the firstoptical window and the second optical window.

By way of example, a system of method 400 may monitor first respectivenumbers of light pulses transmitted toward respective portions of theFOV by the LIDAR device, and second respective numbers of reflectedlight pulses detected by the LIDAR device from the respective portions.In some instances, the system could also monitor numbers oftransmitted/reflected light pulses transmitted through the first opticalwindow (e.g., the first scan) separately from those transmitted throughthe second optical window (e.g., the second scan). Depending on avariety of factors, the system can use the monitored numbers to decideif a particular portion of the FOV is occluded.

In some scenarios, the LIDAR device may fail to detect a reflection of aparticular transmitted light pulse (even when no occlusion is present)for a variety of reasons. In a first scenario, some of the transmittedand/or reflected light pulses may be diverted away from an expectedoptical path due to environmental factors (e.g., small particles in theair, electromagnetic noise, weather conditions, etc.). In a secondscenario, some of the transmitted and/or reflected light pulses may notbe detected due to a fault in the LIDAR device (i.e., not due to anexternal occlusion), such as pulses degraded by a filter of the LIDARdevice, pulses diverted by an aperture of the LIDAR, pulses diverted dueto a misalignment error, or pulses that are not detected due to otherintrinsic characteristics of the LIDAR device (e.g., sensitivity of thedetectors, intrinsic measurement errors, etc.). In a third scenario, oneor more light pulses may be temporarily occluded by an object in the FOV(e.g., a flying piece of paper that occludes the LIDAR device only for ashort period of time, etc.).

Accordingly, in some examples, a system of method 400 may be configuredto determine a likelihood that the occlusion is physically coupled tothe LIDAR device (e.g., attached to the LIDAR device, or attached toanother nearby structure, etc.), the extent of the occlusion, and/orwhether the occlusion is likely to remain physically coupled to theLIDAR device if no responsive action is taken (e.g., without activatinga cleaning apparatus, etc.). For instance, the system can make thesedeterminations by assessing various factors such as: returning lightpulse intensities/numbers, prior information about the prevalence of acertain type of occlusion in a region of the environment where the LIDARdevice is currently located, a speed of a vehicle on which the LIDARdevice is mounted, and/or corroborating data from other sensors, amongother possible factors.

In a first example, method 400 may involve determining whether theocclusion is coupled to the LIDAR device based on at least a speed of avehicle on which the LIDAR device is mounted. For instance, a fastmoving vehicle may be more susceptible (than a slower vehicle) toocclusions like a plastic bag or other flying debris remainingphysically attached to the LIDAR device for a longer period of time(e.g., due to a relatively stronger headwind expected when the vehicleis moving at a high speed).

In a second example, determining whether the occlusion is coupled to theLIDAR device may involve determining whether the occlusion remains at anoffset position relative to the LIDAR device in response to a movementof the LIDAR device from a first location to a second location in anenvironment of the LIDAR device. For instance, where the LIDAR device ismounted on a vehicle (e.g., vehicle 300), a computing system (e.g.,computing system 310) of the vehicle may compare measurements of adistance between the LIDAR device and the occlusion indicated by a firstscan (performed when the LIDAR device is at the first location) and asecond scan (performed when the LIDAR device is at the second location).If the comparison indicates that the occlusion remains at a particularoffset position relative to the LIDAR device, then the computing systemcan determine that the occlusion is physically coupled to the LIDARdevice (e.g., attached to the LIDAR device or attached to the vehicle).Alternatively or additionally, in this example, if the distance to theocclusion changes and/or is no longer present after the vehicle (andthus the LIDAR device) moves from the first location to the secondlocation, then the computing system may determine that the occlusion isnot coupled to the LIDAR device (e.g., a nearby object in theenvironment that is not physically attached to (or near) the LIDARdevice, etc.).

In a third example, method 400 may involve identifying the portion ofthe FOV at block 408 further based on sensor data collected by one ormore other sensors configured to scan the portion of the FOV. Forinstance, vehicle 300 may be configured to scan the same portion of theFOV using the LIDAR device of method 400 (e.g., LIDAR 332) and one ormore other sensors (e.g., LIDAR(s), RADAR(s), camera(s), etc.) of sensorsystem determining whet 304. In this instance, the vehicle may compare(e.g., via computer system 310, etc.) the scans by the LIDAR device withthe scans by the one or more other sensors the multiple sensors todetermine if the LIDAR device is occluded from scanning the portion ofthe FOV identified at block 408.

In some examples, method 400 involves determining an extent to which theocclusion occludes the identified portion of the FOV. In one example,identifying the portion of the FOV at block 408 may be based on adetermination that the occlusion intercepts at least a threshold numberof light pulses. In one example, the threshold number may be 10,000pulses (e.g., in a system that requires detection of at least 10,000pulses from the portion of the FOV, etc.). Other threshold numbers arepossible as well.

In some examples, method 400 may involve determining a first number oflight pulses transmitted by the LIDAR device toward a given portion ofthe FOV and a second number of detected light pulses that reflected offone or more objects in the given portion of the FOV. In some examples,method 400 may also involve identifying a given portion of the FOV thatis at least partially occluded based on the second number of lightpulses being less than half the first number of light pulses. In oneexample, a system of method 400 may determine that the number of lightpulses occluded from the given portion of the FOV is less than a numberof detected light pulses that were reflected by objects in the givenportion of the FOV. In this example, the system may decide that thegiven portion of the FOV is likely not at least partially occluded by agiven occlusion (e.g., the second number of light pulses is notsufficient for identifying the given portion of the FOV as beingpartially occluded, etc.).

In some examples, method 400 involves determining whether the occlusion(of block 408) rotates with the housing. As noted above, in someinstances, a system of method 400 may respond to the detection of anocclusion depending on a type of the occlusion. In one scenario, if theocclusion rotates with the housing, the occlusion might correspond to anobject (e.g., dirt, debris, etc.) that is disposed on an optical window.However, in another scenario, the rotating occlusion might alternativelycorrespond to an object (e.g., plastic sheet, etc.) that is attached toa rotating component of the LIDAR device (e.g., rotating platform 210,housing 250, etc.), where a portion of the attached object extends nearone or both optical windows (e.g., portion of a plastic bag attached tothe housing but extending near one or both of the optical windows.

In some examples, a system of method 400 may be configured to attemptremoving the occlusion (e.g., using cleaning apparatus 160, etc.) inresponse to a determination that the occlusion rotates with housing.

To facilitate determining whether the occlusion rotates with the housing(and/or if the occlusion is disposed on the first optical window), in afirst example, a system of method 400 could determine a first number oflight pulses transmitted through a particular section of the firstoptical window. The particular section of the first optical window canbe selected as the section through which transmitted light pulses withina range of pitch angles where the least partially occluded portion ofthe FOV identified at block 408 is present. Referring back to FIG. 2Dfor instance, the identified portion of the FOV may correspond to theportion of the FOV scanned when mirror 244 is at a pitch angle aboutaxis 215 between 0° and −10°. In this example, the system may computethe first number of light pulses that are transmitted through the firstoptical window 252 (i.e., in the first scan) within this range of pitchangles and at any yaw angle (e.g., any rotational position of platform210 about axis 219). Next, the system may determine a second number ofreflected light pulses that were detected (e.g., by detector 232 ofLIDAR 200) from the same range of pitch angles (and at any yaw angle).The system may then compare the first number and the second number todetermine if the occlusion is likely disposed on the first opticalwindow. For instance, if the second number is less than half the firstnumber (i.e., more light pulses were intercepted by the occlusion atevery yaw angle than light pulses transmitted to the identified portionof the FOV), then the system can determine that the occlusion likelyrotates with the housing (e.g., because it seems to occlude the firstoptical window consistently regardless of the yaw direction of thehousing). Accordingly, in some implementations, method 400 may involvedetermining whether the occlusion rotates with the housing based on thefirst scan.

In a second example, to determine if the occlusion occludes the secondoptical window, the system may determine and compare respective numbersof transmitted and detected light pulses, through the second opticalwindow, that are within the same range of pitch angles (and that haveany yaw angle). Accordingly, in some implementations, method 400 mayinvolve determining whether the occlusion rotates with the housing basedon the second scan.

In a third example, to distinguish between occlusions that occlude onlyone of the two optical windows and occlusions that occlude both windows,the system may perform both of the comparisons described above in thefirst example and the second example. If it is determined that theocclusion occludes only the second optical window but not the firstoptical window for instance, then the system may confirm itsdetermination that the occlusion likely rotates with the housing (andthat it is likely disposed at or near the second optical window).Accordingly, in some implementations, method 400 may involve determiningwhether the occlusion rotates with the housing based on the first scanand the second scan.

In some examples, as noted above, method 400 may involve determiningthat the occlusion does not occlude the identified portion of the FOV(of block 408) in the first scan associated with the first opticalwindow. In these examples, as noted above, method 400 may also involvedetermining that the occlusion is disposed on the second optical windowbased on the determination that the occlusion does not occlude theidentified portion of the FOV in the first scan associated with thefirst window.

In some examples, method 400 involves operating a cleaning apparatusbased on a determination that the occlusion is disposed on the secondoptical window. Referring back to FIG. 1 for example, system 100 mayoperate cleaning apparatus 160 to attempt removing the occlusion fromthe second optical window. For instance, cleaning apparatus 160 mayinclude a liquid spray device, a high pressure air pump, or a wipingdevice, among other examples. In some instances, system 100 may operatethe cleaning apparatus based on a location of the particular section ofthe FOV where the occlusion expected to reside (e.g., the sectionassociated with the range of pitch angles of light pulses transmittedtoward the identified portion of the FOV, etc.). For instance, system100 could operate cleaning apparatus 160 to clean the particular sectionof the second optical window efficiently (e.g., focus air pump or wiperor liquid spray at the location of the particular section of the secondoptical window where the occlusion is expected to be present, etc.).

In some examples, method 400 involves mapping respective portions of theFOV to respective sections of the first optical window. In theseexamples, identifying the portion of the FOV at block 408 may be basedon the mapping. For instance, a system of method 400 may be configuredto monitor multiple sections of the first optical window simultaneouslyto facilitate detecting occlusions that occlude a respective section ofthe first optical window regardless of the yaw orientation of therotating housing.

In a first implementation, the FOV of LIDAR 200 may extend verticallybetween a pitch angle (e.g., angular position of mirror 244 about axis215, etc.) of +30° to a pitch angle of −90°. For instance, LIDAR 200 maybe configured to vertically steer the first plurality of light pulsesthrough the first optical window 252 at respective pitch angles withinthis range (i.e., between +30° and)−90°. In this implementation, thesystem of method 400 (e.g., LIDAR 200, system 100, vehicle 300, etc.)may be configured to map a first portion of the FOV within a pitch rangeof +30° to +10° with a corresponding (lengthwise) section of opticalwindow 252 through which a subset of light pulses 260 having pitchangles within this range are transmitted out of housing 250. Similarly,a second section of optical window 252 could be mapped with a secondportion of the FOV a pitch range of +10° to −10°, and so on. In thisimplementation, 6 lengthwise sections of window 252 can be selected formapping with 6 pitch-wise portions of the FOV.

In a second implementation, the mapping can be alternatively limited toa smaller subset of the entire FOV (e.g., a portion of the vertical FOVbetween angles +10° to −20° can be mapped to six corresponding sectionsof optical window 252, etc.). For instance, the range of pitch anglesselected for the mapping may be selected according to an application ofLIDAR 200. For example, in an application where LIDAR 200 is mounted ona vehicle, the selected range can be a range of pitch angles thatoverlaps expected locations of relatively stationary and/or uniformsurfaces (e.g., road, etc.).

In a third implementation, the respective portions of the FOV can bemapped to any number of sections of the first optical window (e.g., notnecessarily 6 sections), and one or more sections may be assigned to adifferent vertical portion of the FOV than other sections (e.g., firstsection from +10° to −10°, second section from −10° to −15°, etc.). Tothat end, the sizes of each range of pitch angles selected for mappingmay depend on various applications of LIDAR 200. For instance, a smallerrange of pitch angles can be selected in a scenario where detection ofsmall occlusions is appropriate (e.g., LIDAR 200 may be configured todetect objects at a high resolution within that range of pitch angles,etc.).

In some examples, mapping the respective portions of the FOV to therespective sections of the first optical window may involve, for eachsection of the first optical window: identifying emitted light pulsesthat are transmitted toward the section; and identifying reflected lightpulses that are detected by one or more detectors of the LIDAR deviceand that correspond to reflections of the emitted light pulses. In theseexamples, mapping the respective portions may also optionally involvedetermining (based on the identification of the emitted light pulses andthe reflected light pulses) an occlusion score indicative of alikelihood that a respective portion of the FOV associated with thesection is at least partially occluded. The occlusion score may includeany statistical measure of a relationship between the emitted lightpulses and the reflected light pulses.

To that end, in some examples, determining the occlusion score mayinvolve determining a first count of the identified light pulses and asecond count of the reflected light pulses. For example, in line withthe discussion above, a system of method 400 may keep track of the firstcount of emitted light pulses transmitted to each section of the firstoptical window, and the second count of reflected light pulses (e.g.,received by the LIDAR device from a corresponding mapped portion of theFOV illuminated via that section).

In one implementation, the occlusion score may correspond to adifference between the first count and the second count. Other exampleocclusion score computations for determining the likelihood that anocclusion is present are possible as well.

In some examples, method 400 may involve selecting one of a plurality ofcleaning apparatuses (e.g., cleaning apparatuses 160, etc.) forattempting removal of the occlusion based on at least the occlusionscore. In one example, if the occlusion score is relatively high (e.g.,difference between the first count of emitted light pulses and thesecond count of reflected light pulses is greater than a first thresholdscore, etc.), then the system may select a first cleaning apparatus(e.g., wiper, etc.) to attempt removal of the occlusion. In anotherexample, if the occlusion score is relatively low (e.g., differencebetween first count and second count is: (i) less than the firstthreshold score and (ii) greater than a second threshold score), thenthe system may select a second cleaning apparatus (e.g., water spray,etc.) to attempt removal of the occlusion.

In some examples, identifying the reflected light pulses of the mappingmay be based on the reflected light pulses being reflected from one ormore objects that are at greater than a threshold distance to the LIDARdevice. The threshold distance to the LIDAR device (for a reflectedlight pulse to be included in the second count) may be selecteddepending on a variety of factors. In one implementation, the thresholddistance may be approximately zero. In this implementation, a system ofmethod 400 may consider any detected reflection from an object that isout of the housing (i.e., at any distance away from the first opticalwindow) as a successful (not occluded) scan of a respective portion ofthe FOV. In another implementation, the threshold distance can begreater than zero. For example, the system may be configured toassociate detected pulses that were reflected 10 centimeters away (orless) from the LIDAR device with potential occlusions coupled to theLIDAR device (e.g., paper that is attached to vehicle 300 and extendingin front of the first optical window within 10 centimeters away from theLIDAR device, etc.). Other threshold distances are possible.

In some examples, method 400 involves determining that the occlusion (ofblock 408) is disposed on the first optical window based on the mapping.As noted above for example, a system of method 400 may determine thatthe occlusion is disposed on the first optical window if the secondcount of reflected light pulses (e.g., corresponding to transmittedlight pulses that are deemed not occluded by the occlusion) is lowerthan an acceptable level (e.g., less than half the first count oftransmitted level, or less than any other threshold, etc.).

In some examples, method 400 may also involve identifying a particularsection of the first optical window where the occlusion is disposedbased on the mapping. As noted above, for example, the mapping can beused to identify the particular section (of the mapped sections)associated with a relatively low number of reflected light pulsedetections (e.g., relative to the total number of light pulsestransmitted toward the particular section).

In some examples, method 400 may also involve mapping the respectiveportions of the FOV to respective sections of the second optical window.For instance, the same respective portions of the FOV (e.g., ranges ofpitch angles) that are mapped to the first optical window can besimilarly mapped to the second optical window.

Additionally, in these examples, determining that the occlusion isdisposed on the first optical window may be further based on the mappingof the respective portions of the FOV to the respective sections of thesecond optical window. For instance, the system of method 400 mayconfirm that the occlusion is attached to the first optical window only(e.g., rather than an occlusion that affects both optical windows) bychecking if the occluded portion of the FOV also occludes the secondoptical window at any yaw angle in the second scan (as well as in thefirst scan).

In some implementations, a system of method 400 may be configured tomonitor light pulses transmitted toward (and/or received from) aparticular portion of the FOV through the first optical window orthrough the second optical window. For instance, the system couldmonitor the number of successful (not occluded) light pulse detectionsreflected from each respective portion of the FOV relative to the totalnumber of light pulses transmitted to that selected portion (e.g.,regardless of the particular optical window through which the lightpulses were transmitted and/or received).

Accordingly, in some examples, method 400 may involve, for eachrespective portion of a plurality of portions of the FOV: identifyingemitted light pulses transmitted toward the respective portion throughthe first optical window or the second optical window; and identifyingreflected light pulses that are detected by one or more detectors of theLIDAR device and that correspond to reflections of the identified lightpulses. In some examples, identifying the reflected light pulses may bebased on the reflected light pulses being reflected from one or moreobjects that are at a greater than a threshold distance to the LIDARdevice, in line with the discussion above.

In some examples, method 400 may also optionally involve determining anocclusion score (based on the identification of the emitted light pulsesand the reflected light pulses), similarly to the occlusion scoredescribed above for the mapping of the respective sections of the firstoptical window. For example, determining the occlusion score may involvedetermining a first count of the emitted light pulses and a second countof the reflected light pulses.

In this way, for instance, the example system can monitor the respectiveportions of the FOV for occlusions that affect both optical windows. Forexample, the system can determine that a particular portion of the FOVis occluded by an occlusion that does not rotate with the housing if thesecond count is less than half the first count (or if the differencebetween the second count and the first count is more than a threshold).Accordingly, in some implementations, method 400 may involve determiningwhether the occlusion rotates with the housing based on the first countand the second count.

In some examples, method 400 may involve selecting a plurality ofportions of the FOV to monitor for presence of occlusions. In theseexamples, identifying the portion of the FOV at block 408 may involveidentifying the portion from the selected plurality of portions of theFOV. In one example, the selected plurality may correspond to theportions of the FOV that are mapped to sections of the first opticalwindow (and/or the second optical window). In another example, theselected plurality may correspond to the portions of the FOV that aremapped to corresponding (emitted and/or detected) light pulsesregardless of the particular optical window through which these portionsare scanned by the LIDAR device.

In some implementations, each selected portion of the FOV may have asame width and a same height. In some examples, the width may beassociated with a respective range of light pulse yaw directions, andthe height may be associated with a respective range of light pulsepitch directions. In other examples, the width and height of eachportion of the FOV may vary. In some instances, the width and height ofeach selected portion may be associated with a threshold size of theocclusion that the system wants to detect. For example, a larger portionof the FOV can be monitored for detecting larger occlusions, and asmaller portion can be monitored separately to facilitate detectingsmaller occlusions, etc.

Referring back to FIG. 2B, in one embodiment, a first selected portionmay include a first subset of emitted light pulses 260 having a yawangle within a first range of yaw angles (e.g., rotational positions ofplatform 210 about axis 219) between 0° to +30° and having a pitch anglewithin a first range of pitch angles (e.g., rotational positions ofmirror 244 about axis 215) between 0° and −30°. A second selectedportion may be associated with another respective range of light pulseyaw and pitch angles, and so on.

In some examples, method 400 involves obtaining data indicating alocation of a target object in the FOV. In these example, selecting theplurality of portions of the FOV (e.g., to monitor for light pulsetransmissions and corresponding detections, etc.) may be based on thelocation of the target object.

In a first example, the LIDAR device of method 400 can be included in avehicle such as vehicle 300. In this example, the data indicating thelocation of the target can be from another sensor of the vehicle, suchas one or more of the sensors in sensor system 304 (e.g., another LIDAR,a RADAR, a camera, etc.), which scans a second FOV that at leastpartially overlaps with the FOV of the LIDAR device. In this way, forinstance, the selected portions of the FOV can be limited to portionswhere the target object is known to be present (based on the data fromthe other sensor), and the system could thus determine the first andsecond counts of light pulses by monitoring for reflections from justthe portions where the target object is located. For instance, if otherportions of the FOV that do not include the target object were selectedinstead of the portions that include the target object, a failure todetect reflection from those other portions may be caused by the absenceof any objects in the other options (rather than being caused by thepresence of an occlusion that occludes the other portions). Accordingly,in some implementations, obtaining the data indicating the location ofthe target object may involve obtaining sensor data collected by anothersensor configured to scan a second FOV that least partially overlaps theFOV of the LIDAR device.

Alternatively or additionally, in a second example, the data indicatingthe location of the target object could be obtained from a stored map orother representation of an environment of the LIDAR device. For example,data storage 314 of vehicle 300 (and/or data storage 108 of system 100)may store a map of the environment that indicates locations ofrelatively stationary objects (e.g., roads, pavements, walls, etc.) inthe environment of vehicle 300 (and/or the LIDAR device). Thus, the mapof the environment can be used as a source for identifying locations oftarget objects to select particular portions of the FOV to monitor forocclusions (e.g., because reflected light pulses may be expected fromthose particular portions of the FOV). Accordingly, in someimplementations, obtaining the data indicating the location of thetarget object may involve obtaining, from data storage, a storedrepresentation of an environment of the LIDAR device.

In some examples, method 400 may involve obtaining data indicating amounting configuration of the LIDAR device. In these examples, selectingthe plurality of portions of the FOV may be based on at least themounting configuration of the LIDAR device. In some instances, the dataindicating the mounting configuration may be obtained from data storage(e.g., data storage 108); and may pertain to one or more structuressurrounding the LIDAR device.

In a first example, the obtained data may indicate a shape and/orposition of a mount (e.g., stationary platform 212, etc.) that couplesthe LIDAR device with a system (e.g., vehicle 300), platform (e.g.,robotic platform, etc.), or any other structure to which the LIDARdevice is mounted. In this example, a system of method 400 may selectportions of the FOV (e.g., to monitor for presence of occlusions) otherthan particular portions that are expected to be occluded by the mount.

In a second example, the obtained data may indicate a shape and/orposition of one or more structures of a system (e.g., vehicle 300,etc.), platform or any other structure on which the LIDAR device ismounted. For instance, when selecting the portions of the FOV ofvehicle-mounted LIDAR device to monitor for occlusions, an examplesystem of method 400 may include portions of the FOV other than givenportions in which at least part of the vehicle (e.g., vehicle 300) ispresent.

Thus, in the first and second examples above, a system of method 400 mayuse the obtained data to distinguish external occlusions (e.g., externalobject, dust, etc.) that obstruct at least a portion of the scanned FOVfrom nearby structures associated with the mounting configuration of theLIDAR device (e.g., mounting platform, top of vehicle, side of vehicle,etc.). Other examples are possible.

FIG. 5 is a flowchart of another method 500, according to exampleembodiments. Method 500 presents an embodiment of a method that could beused with any of system 100, device 200, vehicle 300, and/or method 400,for example. Method 500 may include one or more operations, functions,or actions as illustrated by one or more of blocks 502-508. Although theblocks are illustrated in a sequential order, these blocks may in someinstances be performed in parallel, and/or in a different order thanthose described herein. Also, the various blocks may be combined intofewer blocks, divided into additional blocks, and/or removed based uponthe desired implementation.

At block 502, method 500 involves rotating a housing of a LIDAR device.For example, block 502 may be similar to block 402 of method 400.

At block 504, method 500 involves transmitting a first plurality oflight pulses through a first optical window of the housing and a secondplurality of light pulses through a second optical window of thehousing. For example, a controller (e.g., controller 104 of system 100)of LIDAR 200 may cause emitter 222 to emit a sequence of light pulses260. While emitter 222 is emitting the sequence of light pulses, thecontroller may also cause actuator 214 to rotate mirror 244 about axis215. In this way, a first subset of the emitted sequence of light pulses260 (i.e., the first plurality of light pulses) may be reflected bymirror 244 toward the first optical window 252 for transmission out ofthe housing 250, and a second subset of the emitted sequence of lightpulses 260 (i.e., the second plurality of light pulses) may be reflectedby mirror 244 toward the second optical 254 for transmission out of thehousing 250.

In some examples, the first optical window may be located at a firstside of the housing and the second optical window may be located at asecond side of the housing opposite the first side. As shown in FIG. 2Cfor example, the first optical window 252 is located at a first side ofhousing 250, and the second optical window 254 is located at a secondside of housing 250 opposite the first side.

At block 506, method 500 involves obtaining a first scan of a FOV of theLIDAR device based on the transmitted first plurality of light pulsesand a second scan of the FOV based on the transmitted second pluralityof light pulses. For example, controller 104 of system 100 may obtainthe first scan based on measurements of reflected light pulses detectedby detector(s) 132 and corresponding to reflections of the transmittedfirst plurality of light pulses (i.e., transmitted through the firstoptical window). Similarly, controller 104 may obtain the second scanbased on measurements of reflected light pulses detected by detector(s)132 and corresponding to reflections of the transmitted second pluralityof light pulses (i.e., transmitted through the second optical window).

At block 508, method 500 involves identifying a portion of the FOV thatis at least partially occluded by an occlusion. For example, block 508may be similar to block 408 of method 400.

In some examples, method 500 involves determining whether the occlusionis coupled to the LIDAR device.

In a first example, a system of method 500 may compare scans obtainedthrough the first optical window and/or the second optical window over apredetermined number of scan periods (and/or a threshold period oftime). In this example, if the identified portion of the FOV remainsoccluded, then the system could determine that the occlusion ispotentially coupled to the LIDAR device (or that the occlusion is likelyto remain coupled to the LIDAR device, etc.).

In a second example, the system may determine that the occlusion iscoupled to the LIDAR device based on a location of the occlusion. Forinstance, if the occlusion is at or near a given location of the firstoptical window (and/or the second optical window), then the system maydetermine that the occlusion is potentially coupled to the LIDAR device(e.g., disposed on or near one or both optical windows, etc.).

In a third example, a system of method 500 may determine that theocclusion is coupled to the LIDAR device based on at least adetermination that the occlusion rotates with the housing (in line withthe discussion at block 408 of method 400).

In a fourth example, a system of method 500 may determine that theocclusion is coupled to the LIDAR device based on a determination thatthe identified portion of the FOV at block 508 is not occluded in thefirst scan of the FOV obtained through the first optical window. Forinstance, if the occlusion affects only one optical window, then thesystem may determine that the occlusion is likely coupled to thatoptical window (e.g., physically attached to the LIDAR device, etc.).

Accordingly, in some implementations, determining whether the occlusionis coupled to the LIDAR device involves determining whether theocclusion (e.g., dust, plastic bag, etc.) is physically attached to theLIDAR device.

In some examples, the LIDAR device may be mounted on a system (e.g.,vehicle 300). In these examples, determining whether the occlusion iscoupled to the LIDAR device may involve determining whether theocclusion is physically attached to the system. For example, if theocclusion remains at a substantially same offset position relative tothe LIDAR device when the system (e.g., vehicle 300) moves from a firstlocation to a second location in the environment, then the system candetermine that the occlusion is potentially coupled to the LIDAR device(e.g., part of an object, such as a plastic bag or other object, thatremains within the FOV of the LIDAR device because the object isphysically attached to the vehicle, etc.).

Accordingly, in some implementations, determining whether the occlusionis coupled to the LIDAR device involves determining whether theocclusion remains at a substantially same offset position relative tothe LIDAR device in response to a movement of the LIDAR device from afirst location to a second location in an environment of the LIDARdevice.

FIG. 6 is a flowchart of yet another method 600, according to exampleembodiments. Method 600 presents an embodiment of a method that could beused with any of system 100, device 200, vehicle 300, and/or methods400-500, for example. Method 600 may include one or more operations,functions, or actions as illustrated by one or more of blocks 602-606.Although the blocks are illustrated in a sequential order, these blocksmay in some instances be performed in parallel, and/or in a differentorder than those described herein. Also, the various blocks may becombined into fewer blocks, divided into additional blocks, and/orremoved based upon the desired implementation.

At block 602, method 600 involves receiving data indicative of a firstscan of a FOV from a LIDAR device. The first scan is based on a firstplurality of light pulses transmitted through a first optical window ofthe LIDAR device. For example, computer system 310 of vehicle 300 mayreceive sensor data collected by a LIDAR (e.g., LIDAR unit 332, LIDAR200, etc.). For instance, the received sensor data at block 602 may beindicative of the first scan described at block 404 of method 400 (e.g.,the first scan may be based on the first plurality of light pulsestransmitted by LIDAR 200 through the first optical window 252).

At block 604, method 600 involves receiving data indicative of a secondscan of the FOV from the LIDAR device. The second scan is based on asecond plurality of light pulses transmitted through a second opticalwindow of the LIDAR device. Continuing with the example above, computersystem 310 may also receive sensor data collected by the same LIDAR(e.g., LIDAR 332, LIDAR 200, etc.). For instance, the received sensordata at block 604 may be indicative of the second scan described atblock 406 of method 400 (e.g., the second scan may be based on thesecond plurality of light pulses transmitted by LIDAR 200 through thesecond optical window 254).

At block 606, method 600 identifying a portion of the FOV that is atleast partially occluded by an occlusion. For example, block 606 may besimilar to block 408 of method 400 and/or block 508 of method 500.

In some examples, method 600 involves generating operation instructionsfor a vehicle that includes the LIDAR device. For example, vehicle 300may include the LIDAR device of method 600 (e.g., as part of LIDAR unit332). Thus, for instance, computer system 310 may use data from theLIDAR device (and/or one or more other sensors of sensor system 304) togenerate the operation instructions for operating vehicle 300. To thatend, the generated operation instructions may relate to any of thefunctions described in connection with control system 306, such asnavigation instructions for navigating vehicle 300 in the environment(e.g., navigation/pathing system 348, obstacle avoidance system 350,etc.), instructions for operating one or more components of sensorsystem 304 (e.g., adjusting the FOVs scanned by the respective sensors,etc.), among other examples.

In some examples, method 600 involves modifying the generated operationinstructions in response to identifying the portion of the FOV at block606 (e.g., the portion that is at least partially occluded). In a firstexample, vehicle 300 may select or assign another sensor of sensorsystem 304 to scan (at least part of) the identified (occluded) portionof the FOV in addition to or instead of the LIDAR device of method 600.In a second example, vehicle 300 may modify navigation instructions(e.g., previously generated for navigating the vehicle in an autonomousmode) to stop the vehicle until an identified occlusion is removed toallow the LIDAR device to scan the identified portion of the FOV. In athird example, vehicle 300 may provide (e.g., via touchscreen 354) amessage for display to a user of vehicle 300 (e.g., the message mayalert the user that the LIDAR device is occluded, etc.). In a fourthexample, vehicle 300 may operate wireless communication system 352 toreport the occlusion to a remote server that processes calibrationand/or maintenance requests for vehicle 300. Other examples arepossible.

Thus, in some examples, a vehicle herein may be configured to operate inan autonomous or semi-autonomous mode based at least in part on sensordata collected by the LIDAR device of method 600. In these examples, oneor more operations of the vehicle can be adjusted to account for thedetection and/or identification of the occlusion in various ways.

In some examples, method 600 involves generating a combined point cloudrepresentation of the first scan and the second scan. For example,computer system 310 may combine the two scans of the FOV (e.g., thefirst scan and the second scan simultaneously obtained via,respectively, the first optical window and the second optical window) togenerate a single 3D point cloud representation having a greater numberof data points (e.g., higher resolution, etc.) as compared to a pointcloud that represents only one of the two scans.

In some examples, method 600 involves providing the combined point cloudrepresentation for display. For example, computer system 310 may cause adisplay (e.g., touch screen 354) to render a 3D view of the scanned FOV(i.e., the combined point cloud representation) for display to a user ofvehicle 300.

FIG. 7 is a flowchart of still another method 700, according to exampleembodiments. Method 700 presents an embodiment of a method that could beused with any of system 100, device 200, vehicle 300, and/or methods400-600, for example. Method 700 may include one or more operations,functions, or actions as illustrated by one or more of blocks 702-706.Although the blocks are illustrated in a sequential order, these blocksmay in some instances be performed in parallel, and/or in a differentorder than those described herein. Also, the various blocks may becombined into fewer blocks, divided into additional blocks, and/orremoved based upon the desired implementation.

At block 702, method 700 involves rotating a housing of a LIDAR device.For example, block 702 may be similar to block 402 of method 400.

At block 704, method 700 involves transmitting a plurality of lightpulses through an optical window of the housing to obtain a scan of aFOV. For example, block 704 may be similar to block 404 of method 400.

At block 706, method 700 involves identifying a portion of the FOV thatis at least partially occluded.

As noted above, in some examples, an example system or device herein maybe configured to detect occlusions based on one or more scans obtainedusing a LIDAR device that transmits light pulses through a singleoptical window (e.g., instead of two separate optical windows). To thatend, identifying the at least partially occluded portion of the FOV atblock 706 may be similar to one or more processes described at blocks408, 508, and/or 606.

By way of example, a system of method 700 may be configured to determinea likelihood that the occlusion is physically coupled to the LIDARdevice (e.g., attached to the LIDAR device or to another nearbystructure, etc.), the extent of the occlusion, and/or whether theocclusion is likely to remain physically coupled to the LIDAR device ifno responsive action is taken (e.g., without activating a cleaningapparatus, etc.), in line with the discussion at blocks 408, 508, and/or606. For instance, method 700 may also involve determining whether theocclusion rotates with the housing, operating a cleaning apparatus,mapping respective portions of the FOV to respective sections of theoptical window, determining an occlusion score for each section of theoptical window, selecting a plurality of portions of the FOV to monitorfor presence of occlusion, and/or obtaining data indicating a locationof a target object in the FOV, among other examples described at blocks408, 508, and/or 606.

IV. CONCLUSION

The particular arrangements shown in the Figures should not be viewed aslimiting. It should be understood that other implementations may includemore or less of each element shown in a given Figure. Further, some ofthe illustrated elements may be combined or omitted. Yet further, anexemplary implementation may include elements that are not illustratedin the Figures. Additionally, while various aspects and implementationshave been disclosed herein, other aspects and implementations will beapparent to those skilled in the art. The various aspects andimplementations disclosed herein are for purposes of illustration andare not intended to be limiting, with the true scope and spirit beingindicated by the following claims. Other implementations may beutilized, and other changes may be made, without departing from thespirit or scope of the subject matter presented herein. It will bereadily understood that the aspects of the present disclosure, asgenerally described herein, and illustrated in the figures, can bearranged, substituted, combined, separated, and designed in a widevariety of different configurations.

What is claimed is:
 1. A method comprising: rotating a housing of alight detection and ranging (LIDAR) device, wherein the housing includesa first optical window and a second optical window; transmitting,through the first optical window, a first plurality of light pulses toobtain a first scan of a field-of-view (FOV) of the LIDAR device;transmitting, through the second optical window, a second plurality oflight pulses to obtain a second scan of the FOV; and based on the firstscan and the second scan, identifying a portion of the FOV that is atleast partially occluded by an occlusion.
 2. The method of claim 1,further comprising determining whether the occlusion is coupled to theLIDAR device.
 3. The method of claim 2, wherein the LIDAR device ismounted to a system, and wherein determining whether the occlusion iscoupled to the LIDAR device comprises determining whether the occlusionis physically attached to the system.
 4. The method of claim 2, whereindetermining whether the occlusion is coupled to the LIDAR devicecomprises determining whether the occlusion is physically attached tothe LIDAR device.
 5. The method of claim 1, further comprisingdetermining whether the occlusion rotates with the housing based on thefirst scan and the second scan.
 6. The method of claim 1, furthercomprising, based on the first scan and the second scan, determiningthat the occlusion does not occlude the identified portion of the FOV inthe first scan associated with the first optical window.
 7. The methodof claim 6, further comprising: based on the determination that theocclusion does not occlude the identified portion of the FOV in thefirst scan, determining that the occlusion is disposed on the secondoptical window; and operating a cleaning apparatus based on thedetermination that the occlusion is disposed on the second opticalwindow.
 8. The method of claim 1, further comprising: mapping respectiveportions of the FOV to respective sections of the first optical window,wherein identifying the portion of the FOV is based on the mapping. 9.The method of claim 8, wherein mapping the respective portions of theFOV comprises, for each section of the first optical window: identifyingemitted light pulses that are transmitted toward the section of thefirst optical window; and identifying reflected light pulses that aredetected by one or more detectors of the LIDAR device and thatcorrespond to reflections of the identified light pulses.
 10. The methodof claim 9, further comprising, for each section of the first opticalwindow: based on at least the identification of the emitted light pulsesand the identification of the reflected light pulses, determining anocclusion score indicative of a likelihood that a respective portion ofthe FOV associated with the section is at least partially occluded. 11.The method of claim 10, further comprising: based on at least theocclusion score, selecting one of a plurality of cleaning apparatusesfor attempting removal of the occlusion.
 12. The method of claim 9,wherein identifying the reflected light pulses is based on the reflectedlight pulses being reflected from one or more objects that are atgreater than a threshold distance to the LIDAR device.
 13. The method ofclaim 8, further comprising determining that the occlusion is disposedon the first optical window based on the mapping.
 14. The method ofclaim 8, further comprising: mapping the respective portions of the FOVto respective sections of the second optical window, wherein determiningthat the occlusion is disposed on the first optical window is furtherbased on the mapping of the respective portions of the FOV to therespective sections of the second optical window.
 15. The method ofclaim 1, further comprising, for each respective portion of a pluralityof portions of the FOV: identifying emitted light pulses transmitted,through the first optical window or the second optical window, towardthe respective portion of the FOV; and identifying reflected lightpulses that are detected by one or more detectors of the LIDAR deviceand that correspond to reflections of the identified light pulses. 16.The method of claim 15, further comprising, for each respective portionof the plurality of portions of the FOV: based on at least theidentified emitted light pulses and the identified reflected lightpulses, determining an occlusion score indicative of a likelihood thatthe respective portion is at least partially occluded.
 17. The method ofclaim 1, further comprising: selecting a plurality of portions of theFOV to monitor for presence of occlusions, wherein identifying theportion of the FOV that is at least partially occluded comprisesidentifying the portion of the FOV from the selected plurality ofportions of the FOV.
 18. The method of claim 17, further comprising:obtaining data indicating a location of a target object in the FOV,wherein selecting the plurality of portions of the FOV is based on thelocation of the target object.
 19. The method of claim 18, whereinobtaining the data indicating the location of the target objectcomprises obtaining sensor data collected by another sensor configuredto scan a second FOV that at least partially overlaps the FOV of theLIDAR device.
 20. The method of claim 17, further comprising: obtaining,from data storage, an indication of a mounting configuration of theLIDAR device, wherein selecting the plurality of portions of the FOV isbased on the mounting configuration of the LIDAR device.